Lens assembly with integrated feedback loop and time-of-flight sensor

ABSTRACT

This invention provides an integrated time-of-flight sensor that delivers distance information to a processor associated with the camera assembly and vison system. The distance is processed with the above-described feedback control, to auto-focus the camera assembly&#39;s variable lens during runtime operation based on the particular size/shape object(s) within the field of view. The shortest measured distance is used to set the focus distance of the lens. To correct for calibration or drift errors, a further image-based focus optimization can occur around the measured distance and/or based on the measured temperature. The distance information generated by the time-of-flight sensor can be employed to perform other functions. Other functions include self-triggering of image acquisition, object size dimensioning, detection and analysis of object defects and/or gap detection between objects in the field of view and software-controlled range detection to prevent unintentional reading of (e.g.) IDs on objects outside a defined range (presentation mode).

RELATED APPLICATION

This application is a continuation of co-pending U.S. patent applicationSer. No. 15/857,733, entitled LENS ASSEMBLY WITH INTEGRATED FEEDBACKLOOP AND TIME-OF-FLIGHT SENSOR, filed Dec. 29, 2017, which is acontinuation-in-part of co-pending U.S. patent application Ser. No.13/800,055, entitled LENS ASSEMBLY WITH INTEGRATED FEEDBACK LOOP FORFOCUS ADJUSTMENT, filed Mar. 13, 2013, the teachings of each of whichapplications are expressly incorporated herein by reference.

FIELD OF THE INVENTION

This application relates to cameras used in machine vision and moreparticularly to automatic focusing lens assemblies and range finders incameras.

BACKGROUND OF THE INVENTION

Vision systems that perform measurement, inspection, alignment ofobjects and/or decoding of symbology (e.g. bar codes, or more simply“IDs”) are used in a wide range of applications and industries. Thesesystems are based around the use of an image sensor, which acquiresimages (typically grayscale or color, and in one, two or threedimensions) of the subject or object, and processes these acquiredimages using an on-board or interconnected vision system processor. Theprocessor generally includes both processing hardware and non-transitorycomputer-readable program instructions that perform one or more visionsystem processes to generate a desired output based upon the image'sprocessed information. This image information is typically providedwithin an array of image pixels each having various colors and/orintensities. In the example of an ID reader, the user or automatedprocess acquires an image of an object that is believed to contain oneor more IDs. The image is processed to identify ID features, which arethen decoded by a decoding process and/or processor to obtain theinherent information (e.g. alphanumeric data) that is encoded in thepattern of the ID.

Often, a vision system camera includes an internal processor and othercomponents that allow it to act as a standalone unit, providing adesired output data (e.g. decoded symbol information) to a downstreamprocess, such as an inventory tracking computer system or logisticsapplication. It is often desirable that the camera assembly contain alens mount, such as the commonly used C-mount, that is capable ofreceiving a variety of lens configurations. In this manner, the cameraassembly can be adapted to the specific vision system task. The choiceof lens configuration can be driven by a variety of factors, such aslighting/illumination, field of view, focal distance, relative angle ofthe camera axis and imaged surface, and the fineness of details on theimaged surface. In addition, the cost of the lens and/or the availablespace for mounting the vision system can also drive the choice of lens.

An exemplary lens configuration that can be desirable in certain visionsystem applications is the automatic focusing (auto-focus) assembly. Byway of example, an auto-focus lens can be facilitated by a so-calledliquid lens assembly. One form of liquid lens uses two iso-densityliquids—oil is an insulator while water is a conductor. The variation ofvoltage passed through the lens by surrounding circuitry leads to achange of curvature of the liquid-liquid interface, which in turn leadsto a change of the focal length of the lens. Some significant advantagesin the use of a liquid lens are the lens' ruggedness (it is free ofmechanical moving parts), its fast response times, its relatively goodoptical quality, and its low power consumption and size. The use of aliquid lens can desirably simplify installation, setup and maintenanceof the vision system by eliminating the need to manually touch the lens.Relative to other auto-focus mechanisms, the liquid lens has extremelyfast response times. It is also ideal for applications with readingdistances that change from object-to-object (surface-to-surface) orduring the changeover from the reading of one object to anotherobject—for example in scanning a moving conveyor containing differingsized/height objects (such as shipping boxes). In general, the abilityto quickly focus “on the fly” is desirable in many vision systemapplications.

A recent development in liquid lens technology is available fromOptotune AG of Switzerland. This lens utilizes a movable membranecovering a liquid reservoir to vary its focal distance. A bobbin exertspressure to alter the shape of the membrane and thereby vary the lensfocus. The bobbin is moved by varying the input current within a presetrange. Differing current levels provide differing focal distances forthe liquid lens. This lens can provide a larger aperture for use invarious applications. However, due to thermal drift and other factors,there may be variation in calibration and focus setting during runtimeuse, and over time in general. A variety of systems can be provided tocompensate and/or correct for focus variation and other factors.However, these can require processing time (within the camera's internalprocessor) that slows the lens' overall response time in coming to a newfocus. It is recognized generally that a control frequency of at leastapproximately 1000 Hz may be required to adequately control the focus ofthe lens and maintain it within desired ranges. This poses a burden tothe vision system's processor, which can be based on a DSP or similararchitecture. That is vision system tasks would suffer if the DSP werecontinually preoccupied with lens-control tasks.

Additionally, in many vision system applications, such as ID-decoding inlogistics operations (e.g. tracking IDs on packages as they pass down aconveyor line), the height, length, overall size and spacing gap betweenobjects is highly variable. This presents challenges for the visionsystem. Various techniques allow for 3D imaging, but they may not becost-effective or suitable for a logistics (or similar) environmentwhere a main goal is detecting presence of a surface containing an IDand acquiring a decodable image of the ID as rapidly and efficiently aspossible. Moreover, ID readers can also be configured in a so-calledpresentation mode, in which the reader is fixedly mounted and typicallydirected to image a scene downwardly. In operation, a user locates an IDa code, typically located at the top surface of an object, within thereading range of the reader (i.e. presenting the ID to the reader). Insuch application the reading range should be clearly defined, but nottoo large, so as to prevent unintentional reading of the code at timesnot desired by the user (and only when deliberately presented).

SUMMARY OF THE INVENTION

This invention overcomes disadvantages of the prior art by providing aremovably mountable lens assembly for a vision system camera thatincludes an integral auto-focusing, liquid lens unit, in which the lensunit compensates for focus variations by employing a feedback controlcircuit that is integrated into the body of the lens assembly. Thefeedback control circuit receives motion information related to andactuator, such as a bobbin (which variably biases the membrane undercurrent control) of the lens from a position sensor (e.g. a Hall sensor)and uses this information internally to correct for motion variationsthat deviate from the lens setting position at a target lens focaldistance setting. The defined “position sensor” can be a single (e.g.Hall sensor) unit or a combination of discrete sensors located variouslywith respect to the actuator/bobbin to measure movement at variouslocations around the lens unit. Illustratively, the feedback circuit canbe interconnected with one or more temperature sensors that adjust thelens setting position for a particular temperature value. In addition,the feedback circuit can communicate with an accelerometer that sensesthe acting direction of gravity, and thereby corrects for potential sag(or other orientation-induced deformation) in the lens membrane basedupon the spatial orientation of the lens.

This invention further provides an integrated (e.g. inexpensive andcommercially available) single-point or multi-point time-of-flightsensor that delivers distance information to a processor associated withthe camera assembly and vison system. This distance information isprocessed, illustratively, in conjunction with the above-describedfeedback control, to auto-focus the camera assembly's variable (e.g.liquid) lens during runtime operation based on the particular size/shapeobject(s) within the field of view. Illustratively, the shortestmeasured distance is used to set the focus distance of the lens. Tocorrect for calibration or drift errors, a further image-based focusoptimization can occur around the measured distance and/or based on themeasured temperature. The distance information generated by thetime-of-flight sensor can also be employed to perform other functions.Another function is self-triggering of image acquisition of an object bythe vision system. That is, when a change in object distance is measured(for example, the change between a positive height and the supportingbase/conveyor height), an image capture is triggered. This function canbe used to trigger acquisition in either presentation mode in which anobject is passed into the camera's field of view, or in conveyor beltapplications in which objects pass through the camera field of viewalong a particular travel direction. Another function is object (e.g.box) size dimensioning in which a vision system with a time-of-flightsensor is mounted above the supporting/measurement base (for example, aconveyor belt). During calibration, the height between measurement baseand vision system is measured and stored in the system. At runtime, thevision system captures images and measures the distance to the object inthe center of the image. If a (e.g.) rectangular shape is detected whichincludes the center of the image, the dimensions of that rectangle aredetermined based on the measured distance and the known opticalproperties of imager (sensor size, focal length). The height of theexemplary box is the difference between the height of the camera andmeasured shortest distance between camera and box.

Another possible function is the detection and analysis of objectdefects. After a detection of the rectangular top surface as describedabove, deviations from the rectangular shape are measured and damagedobjects (e.g. boxes) can be detected. Yet another possible function isregion of interest (RoI) detection. The field of view of a camera-basedID reader is also imaged onto the sensed area of a multi-point (i.e. n×1or n×m, time-of-flight points/pixels) sensor array. The measured 3Dheight map generated by the sensor array can be used to narrow theregion of interest for ID decoding. That is, knowing in which part ofthe image the object resides reduces decoding time as ID candidatefeatures are searched from a narrowed region of interest in the overallacquired image.

Another possible function is gap detection between objects in the fieldof view, which assists in linking the appropriate ID code to theappropriate imaged object (box). In logistics applications, where thereis more than one box in residing within field of view at the same time,the time-of-flight measurement can assist in locating the edge(s) of theobject and to determine what ID is actually associated therewith.

Yet another possible function of the vision system arrangement hereinis, employ the distance measured received from the time-of-flight sensorto limit the reading range of the vision system so as to preventunintentional reading. This can be part of a so-called presentation modefunctionality within the vision system. By way of example, only if thedistance to the object is within a defined range, then (e.g.) the IDreader (a) captures an image, (b) initiates an ID-de decoding process,and/or (c) transmits the decoded data to the host system. If thedistance to the ID code (as indicated by the time-of-flight sensor isoutside this range, then at least one of the above steps (a)-(c) in theoverall ID-reading process can be disabled so a result is not generatedor stored for downstream use in a task, etc.

In an illustrative embodiment, an image-based ID code reader isprovided. The reader comprises a vision system camera with an imagesensor and optics and a vision system processor, which is arranged tofind and decode IDs in acquired images of the vision system camera. Atime-of-flight sensor is integrated with the vison system camera, whichreads distance with respect an object in a field of view of the visionsystem camera. The time-of-flight sensor is operatively connected withat least one of the vision system processor and a camera control. Inembodiments, a change in read object distance can trigger at least oneof image acquisition by the vision system camera and an ID decodingprocess by the vision system processor. Illustratively, the change indistance comprises a reduced distance from a baseline distance derivedfrom a support or conveyor for the object. The time-of-flight sensor canbe a single-point time-of-flight sensor or a multi-point time-of-flightsensor. In an exemplary embodiment, in which the reader operates inso-called presentation mode, at least one of (a) the image acquisition(b) the ID decoding process, and (c) delivery of results from the IDdecoding process (which can be performed by the image sensor, processorand/or a camera control) is/are enabled exclusively if the object iswithin a predetermined distance range.

In another illustrative embodiment, a vision system is provided, whichincludes a 2D image sensor and an imager lens that projects receivedlight from a scene onto the image sensor. The imager lens includes avariable lens with an electrically controllable focus distance. Atime-of-flight sensor receives a distance measurement from the scene,and a temperature sensor generates a temperature measurement withrespect to an ambient environment of the imager lens. A variable lenscontrol circuit is also arranged to set the focus distance of thevariable lens based on the distance measurement and the temperaturemeasurement. Illustratively, the time-of-flight sensor is a single-pointtime-of-flight sensor, and/or the variable lens assembly comprises amembrane-based liquid lens assembly.

In another illustrative embodiment, a vision system is provided, whichincludes a 2D image sensor that acquires images of a scene, an imagerlens that projects received light from the scene onto the image sensor,and a time-of-flight sensor that a receives a distance measurement fromthe scene. A processor is arranged to receive distance data from thetime-of-flight sensor. Based upon the distance data the processordetermines one or more height measurements with respect to one or moreobjects within the scene, so as to perform an analysis process on theone or more objects. Illustratively, the one or more objects define arectilinear shape and the analysis process defines measuring dimensionsof the one or more objects by (a) measuring a distance between the imagesensor and the object in the center of the image; (b) searching for arectangular shape in the center of an image acquired by the image sensorwith one or more vision system tools, (c) computing dimensions of a topsurface of the one or more objects relative to the rectangular shape,based on the measured distance and known optical properties of imagesensor and imager lens, and (d) calculating a height of the one or moreobjects, based on a measured shortest distance between the image sensorand the top surface and a known height position of the image sensorbased upon a reference surface. The reference surface can comprise amoving conveyor (or other relatively flat/planar moving or stationarystage) that passes the one or more objects through the scene. Theprocessor can be arranged to determine deviations from the rectilinearshape to determine a defect in the one or more objects. Also, the one ormore objects can be in relative motion with respect to the image sensorand the time-of-flight sensor. The image sensor and the imager lens canbe mounted so as to acquire images of the one or more objects as theyare transported on a conveyor through a scene. The sensor and lens areoperatively connected to an ID decoder that finds and decodes IDinformation on the one or more objects. The time-of-flight sensor can bea single-point time-of-flight sensor or a multi-point time-of-flightsensor. Illustratively, the processor receives a plurality of heightvalues concurrently from the multi-point time-of-flight sensor relativeto discrete parts of the scene, and based on the measured height valuesdefines at least one region of interest with respect to the one or moreobjects. An ID decoder can receive image data from the scene, and locateand decode ID information within at least one region of interest.Additionally, the ID decoder can receive image data from the scene andlocate and decode ID information from the image data. The processor isarranged to detect a plurality of objects based on the height values inthe scene and to associate one or more located IDs relative to each ofthe plurality of objects, respectively. The imager lens can also definea removable lens assembly that includes an electrically controlledvariable lens and a lens controller circuit operatively connected to thevariable lens. As such, the lens controller circuit can be housed in theremovable lens assembly. In embodiments, the time-of-flight sensor canalso be contained integrally within the removable lens assembly andinterconnected to the lens controller circuit.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention description below refers to the accompanying drawings, ofwhich:

FIG. 1 is a perspective view of the external structure of anexchangeable auto-focus lens assembly with integratedfeedback-loop-based focus control according to an illustrativeembodiment;

FIG. 2 is a side cross section of the lens assembly of FIG. 1 showingthe layout of internal mechanical, optical, electro-optical andelectronic components;

FIG. 3 is a perspective view of the lens assembly of FIG. 1 with outercap removed to reveal the arrangement of components;

FIG. 4 is a perspective view of the lens assembly of FIG. 1 with theouter cap and spacer assembly removed to reveal the interconnectionbetween the liquid lens and the control circuit;

FIG. 5 is a block diagram of the generalized electrical connection anddata flow between the liquid lens, integrated controller and cameravision system processor for the lens assembly of FIG. 1;

FIG. 5A is a flow diagram of a feedback loop-based bobbin positioncontrol process for the lens assembly of FIG. 1;

FIG. 6 is a block diagram of the stored data in the control circuitmemory of FIG. 5;

FIG. 7 is a temperature correction process that generatestemperature-corrected bobbin position values for use with the controlcircuit of FIG. 5 and process of FIG. 5A;

FIG. 8 is a diagram of a vision system, having an integratedtime-of-flight sensor, acquiring an image and concurrent distancemeasurement of an exemplary object according to an embodiment;

FIG. 9 is a flow diagram of a procedure for auto-focus of a variable(e.g. liquid) lens using the integrated time-of-flight sensor in theexemplary vision system of FIG. 8;

FIG. 10 is a flow diagram of a procedure for self-triggering imageacquisition based on presence of an object with respect to theintegrated time-of-flight sensor in the exemplary vision system of FIG.8;

FIG. 11 is a flow diagram of a procedure for determining size of anobject, and optionally object defects, using the integratedtime-of-flight sensor in the exemplary vision system of FIG. 8;

FIG. 12 is a diagram of a vision system, having an integrated,multi-point time-of-flight sensor, acquiring an image and matrix ofassociated distance measurements of a plurality of objects within afield of view of the vision system camera according to an exemplaryembodiment;

FIG. 13 is a flow diagram of a procedure for determining one or moreregion(s) of interest within an acquired image for (e.g.) finding anddecoding ID features using information provided by the matrix ofdistance measurements of the exemplary vision system of FIG. 12;

FIG. 14 is a flow diagram of a procedure for determining gaps betweenobjects within an acquired image using information provided by distancemeasurements of the exemplary vision system of FIG. 8 or FIG. 12; and

FIG. 15 is a flow diagram of a procedure for determining if an object ispresented to the vision system based upon location of the object withina predetermined range, such as implemented in presentation mode.

DETAILED DESCRIPTION I. Vision System Camera Lens

FIG. 1 details the external structure of an exchangeable, auto-focuslens assembly (also simply termed “lens assembly”) 100 according to anillustrative embodiment. The lens assembly includes an outer cap 110defining a generally cylindrical shape. This outer cap 110 provides aprotective and supporting shell for a variable focus lens element(comprising an Optotune membrane-based liquid lens model EL-6-18 orEL-10-30 in this exemplary embodiment) 120. By way of useful backgroundinformation the present data sheet with specifications for variousmodels of this lens is available on the World Wide Web atwww.optotune.com/images/products/Optotune%20EL-6-18.pdf. It is expresslycontemplated that the teachings of the embodiments herein can be appliedto a variety of electronically focused lens types including other formsof liquid lens technology and electro-mechanically adjusted solidlenses. For the purposes of this description, the variable focus lenselement 120 (also simply termed the “liquid lens”) of the overallauto-focus lens assembly 100 is assumed to operate based uponpredetermined inputs of current (or voltage in alternate arrangements),and provides various outputs that the user can employ to monitor andcontrol the lens using conventional techniques. Such outputs can includethe position of the bobbin using, for example, one or more Hall sensors(described further below) and/or the present temperature of the lensusing one or more conventional temperature sensors.

By way of further background, it has been observed that such liquidlenses exhibit excessive drift of its optical power over time andtemperature. Although the lens can be focused relatively quickly to anew focal position (i.e. within 5 milliseconds), it tends to drift fromthis focus almost immediately. The initial drift (or “lag”) is caused bylatency in the stretch of the membrane from one focus state to thenext—i.e. the stretch takes a certain amount of time to occur. A seconddrift effect with a longer time constant is caused by the powerdissipation of the lens' actuator bobbin heating up the lens membraneand liquid. In addition the orientation of the lens with respect to theacting direction of gravity can cause membrane sag that has an effect onfocus. The system and method of the embodiments described herein addressdisadvantages observed in the operation and performance such liquidlenses.

The rear 130 of the lens assembly 100 includes a threaded base that canbe adapted to seat in a standard camera mount, such as the popular cineor (C-mount). While not shown, it is expressly contemplated that thelens assembly 100 can be (removably) mounted a variety of camera typesadapted to perform vision system tasks with an associated vision systemprocessor.

With further reference also to FIGS. 2-4, the construction of the lensassembly 100 is described in further detail. It is expresslycontemplated that the depicted construction is illustrative of a rangeof possible arrangements of components that should be clear to those ofskill in the art. The cap 110 defines a metal shell (for examplealuminum alloy) that includes a side skirt 140 and unitary front face150. The cap overlies a spacer/filler 210 (see also FIG. 3). This filler210 includes a pair of threaded holes 310 (FIG. 3) that receive threadedfasteners 160 to removably secure the cap over the filler 210. A pair ofopposing threaded fasteners 170 are recessed in corresponding holes 172of the cap and pass through holes 320 in the filler 210 and intothreaded holes 410 (FIG. 4) on two keys 440 that rotatably engage themain lens barrel assembly 220 (FIGS. 2 and 4). This relationship isdescribed further below. These fasteners 170 maintain the main lensbarrel assembly 220 in axial alignment with the filler 210.

As shown in FIG. 2, the lens barrel assembly 220 contains a series offixed lenses 230, 232, 234, 236 and 238 arranged according to ordinaryoptical skill behind the liquid lens 210. These lenses allow the imageprojected along the optical axis OA to the vision system sensor to besized appropriately to the sensor's area over a range of varying focaldistances specified for the lens assembly. By way of example, the rangeof optical power can be −2 to +10 diopter. The lenses 230-238 arearranged in a compressed stack within the main barrel assembly 220 withappropriate steps and/or spacers therebetween. The overall stack is heldin place by a threaded retaining ring 240 at the rear end (130) of thelens assembly 110. At the front of the main barrel is located anaperture stop disc 250 that reduces the system aperture to anappropriate, smaller diameter. This allows customization ofbrightness/exposure control and/or depth of field for a given visionsystem application.

The main barrel assembly 220 includes a rear externally threaded base260 having a diameter and thread smaller than that of a C-mount—forexample a conventional M-12 mount size for interchangeability withcamera's employing this standard, or another arbitrary thread size. Athreaded mount ring 262 with, for example, a C-mount external thread 264is threaded over the base thread 260. This ring 262 allows the backfocus of the lens with respect to the camera sensor to be accuratelyset. In general, the shoulder 266 of the ring is set to abut the face ofthe camera mount when the lens is secured against the camera body. Apair of set screws 360 (FIGS. 3 and 4) pass through the ring 262, andremovably engage the base thread 260 to maintain the mount ring 262 atan appropriate back focus setting.

An O-ring 267 is provided on the front face of the liquid lens 120 tocancel out tolerances. In addition, and with reference also to FIG. 4,filler 210 is adapted to rotate with respect to the main barrel assembly220. A pair of semi-circular keys 440, held together by an O-ring 450engage a groove in the filler 210 and allow the filler 210 and cap 110to rotate with respect to the barrel assembly 220 about the axis OA,while fixing these components along the axial direction. In this manner,after the lens assembly threaded base is properly seated in the camerahousing with desired back focus, the cap is rotated to align the cable270 with the camera's connecting socket. This rotation is secured viathe knob 180 (FIG. 1) that threads through a hole 380 in the filler 210and can be tightened to bear against the barrel assembly 220, therebyrotationally locking these components together at the desired rotationalorientation therebetween.

As shown in FIG. 3, the front end of the filler 210 includes a somewhatrectangular recess 330 to support the shape of the liquid lens 120 in aposition at the front of the assembly and in front of the main barrelassembly 220. The filler 210 also includes a flattened top end (shelf)340 with appropriate raised retaining tabs 342 to support a lens controlcircuit board 350 according to an illustrative embodiment. Thearrangement of the shelf 340, circuit board 350 and cap 110 define asufficient gap G (FIG. 2) between the inner surface of the cap and thecircuit board to provide clearance for the board. In an embodiment, theapproximate diameter of the cap is approximately 32 millimeters.

Notably, the barrel assembly 220 is an interchangeable component so thatdifferent fixed lens arrangements can be provided in the overall lensassembly (i.e. with the same liquid lens, cap and control circuitry).Thus, this design provides substantial versatility in providing a rangeof possible focal distances for different vision system applications.

Also notably, the provision of a lens control circuit within the overallstructure of the lens assembly allows certain control functions to belocalized within the lens itself. This is described in further detailbelow. The circuit board 350 is connected via a connector 422 andstandard ribbon cable 420 to the liquid lens 120 as shown in FIG. 4. Thefiller 210 provides a gap to run the cable 420 between these components.Additionally, the control circuit board 350 is connected to a cable 270and multi-pin end connector 272. These are arranged to electricallyconnect to a receptacle on the camera housing (typically along its frontface adjacent to the lens mount). This cable provides power to the lensassembly (the circuit board and liquid lens) from the camera body, andalso provides a data interconnect between the lens and the camera'svision system processor, as described in further detail below. A cutout274 at the rear edge of the cap 110 provides a chase for the cable 270to pass from the interior to the exterior of the lens assembly 110.Appropriate seals and/or close-tolerance fits prevent incursion ofmoisture or contaminants from the environment.

II. Lens Feedback Control

The control functions of the circuit board 350 are now described infurther detail with reference to FIG. 5. As described above, it has beenobserved that the drift or lag can be controlled by measuring theposition of the actuator and the temperature of the lens and using thisdata to control the current through the lens actuator bobbin (a magneticcoil that compresses the lens variable under different currentsettings). In an illustrative embodiment, such drift/lag is compensatedby a control circuit 510 (also termed simply “controller”) on thecircuit board that integrates a (digital) feedback loop completely intothe lens barrel of the lens assembly avoiding the use of the camera'svision system processor to control these adjustments. The controlcircuit includes an associated memory (e.g. an EEPROM) 512 that, asshown in FIG. 6 can be divided into data memory 610 and program memory620. As described further below, the data memory 610 can includecorrection parameters for temperature 612, orientation with respect togravity 614, and other appropriate parameters 616. Such other parameters616 can include tolerance control parameters, such as the flangetolerance correction (described below). The program memory can includethe feedback-loop control software and correction application 622.

At startup, the vision system 520 communicates to the lens assemblycircuit 350 the tolerance value of its flange-to-sensor distance. Thisvalue is the deviation from the ideal C-mount distance (typically 17.526millimeters), which has been measured after assembly of the visionsystem and has been stored in the memory 526 (e.g. a non-volatile flashmemory) of the vision system. The control circuit 510 is arranged tocorrect for the flange tolerance as described further below.

Upon startup, the control circuit 510 can request the vision systemprocessor 522 of the vision system camera 520 to provide the latestfirmware upgrade 528 so that the function lens assembly is synchronizedwith the software and firmware of the vision system. If the firmware isup-to-date, then the processor indicates this state to the lens controlcircuit and no upgrade is performed. If the firmware is out-of-date,then the new firmware is loaded in the appropriate location of the lensassembly's program memory 620 (FIG. 6). This communication typicallyoccurs over the lens assembly's I2C communication interface (531)transmitted over the cable 270 (FIG. 2).

Note, as used herein the terms “process” and/or “processor” should betaken broadly to include a variety of electronic hardware and/orsoftware based functions and components. Moreover, a depicted process orprocessor can be combined with other processes and/or processors ordivided into various sub-processes or processors. Such sub-processesand/or sub-processors can be variously combined according to embodimentsherein. Likewise, it is expressly contemplated that any function,process and/or processor herein can be implemented using electronichardware, software consisting of a non-transitory computer-readablemedium of program instructions, or a combination of hardware andsoftware.

The control circuit 510 can be implemented using a variety of electronichardware. Illustratively a microcontroller is employed. The controlcircuit 510 receives focus information 530 (e.g. focal distance, whichis translated by the controller into target bobbin position) from thevision system camera 520 (i.e. via cable 270 and interface link 531).This focus information can be derived from a focus process 532 thatoperates in the camera processor 522. The focus process can useconventional or custom auto-focus techniques to determine proper focus.These can include range-finding or stepping through a series of focusvalues in an effort to generate crisp edges in the image 534 of anobject acquired by the sensor 536. While highly variable a 2K×1K-pixelsensor is used in the exemplary embodiment. Alternatively, the focusprocess can include data derived from a range-finding sensor, such anintegrated time-of-flight sensor as described below.

The focus information 530 is used by the control circuit 510 to generatea target bobbin position and to provide a digital signal with movementinformation 540 to the current controller 544. The current controllerapplies the appropriate current to an annular bobbin assembly 550 (or“bobbin”), which thereby deforms the liquid lens membrane 552 to providean appropriate convex shape to the bulged lensmatic region 554 withinthe central opening of the bobbin 550. The bobbin 550 includes a magnet558 that passes over a conventional linear Hall sensor 560. This Hallsensor 560 generates a digital position signal 562 that is directed backto the control circuit 510 where it is analyzed for actual bobbinposition (for example, calling up values in the memory 512) versus thetarget position represented by a corresponding Hall sensor targetposition. If, in a comparison of the actual Hall sensor value and targetHall sensor value, these values do not match, then the control circuit510 applies a correction, and that is delivered to the currentcontroller 544, where it is used to move the bobbin 550 to a correctposition that conforms with the target Hall sensor position. Once thebobbin 550 is at the correct position, the controller can signal thatcorrection is complete.

Note that additional Hall sensors (or other position-sensing devices)566 (shown in phantom) can generate additional (optional) positionsignals 568 that are used by the control circuit to verify and/orsupplement the signal of sensor 560. In an embodiment, data istransmitted between components using an I2C protocol, but otherprotocols are expressly contemplated. In general, the commerciallyavailable Hall sensor operates in the digital realm (i.e. using the I2Cinterface protocol), thereby effectively avoiding signal interferencedue to magnetic effects. By way of non-limiting example, a model AS5510Hall linear sensor (or sensors) available from AustriaMicrosystems (AMS)of Austria can be used.

With reference to FIG. 5A, a bobbin position-sensing/correcting feedbackloop process 570 is shown in a series of flow-diagram process steps. Atarget focus distance is received from the vision system processor instep 572. The control feedback loop 570 then initiates as this focusdistance is used by the lens assembly control circuit (controller) 510to determine a target value for bobbin position represented by a targetHall sensor value provided by one or more sensors on the bobbin. Thetarget Hall sensor value(s) can be corrected based upon storedparameters in memory 512 (step 574). Such parameters include, but arenot limited to temperature, spatial orientation andflange-to-sensor-distance tolerance, and this (optional) process isdescribed further below. In step 576, the control circuit 510 measuresthe actual position of the bobbin based upon the position of the Hallsensor(s) and associated signal value(s) (562). In step 578, the controlcircuit 510 then compares the actual, returned Hall sensor value(s) withthe target value. If the values are not substantially equal thendecision step 580 branches to step 582 and the control circuit directsthe current controller 544 to input a current that will move the bobbinto the corrected position. This can be based on the difference incurrent needed to move the bobbin between the actual and correctposition. If the comparison in step 578 determines that the actual andtarget Hall sensor value(s) are substantially equal, then the decisionstep 580 branches to step 582 and the system indicates that correctionis complete. The control circuit repeats correction steps 574, 576, 578,580 and 582 until the actual and target Hall sensor values aresubstantially equal (within an acceptable tolerance), and the newcorrect bobbin position is indicated. This complete status can bereported to the camera's processor 522 for use in performing imageacquisition.

Note that this local feedback loop 570 can run continuously to maintainfocus at a set position once established, and until a new bobbinposition/focus is directed by the camera. Thus, the feedback loop 570ensures a steady and continuing focus throughout the image acquisitionof an object, and does so in a manner that avoids increased burdens onthe camera's vision system processor.

The determination of the target value for the Hall sensor(s) in step 574can include optional temperature, spatial orientation and/or otherparameter (e.g. flange distance) correction based upon parameters 612,614, 616 (FIG. 6) stored in memory 512. Temperature of the lens unit issensed (optionally) by an on-board or adjacent temperature sensor 588(FIG. 5). The temperature sensor 588, like other components of thecircuit 350, can employ a standard interface protocol (e.g. I2C).

As shown in FIG. 7, an optional temperature compensation process 700operating within the control circuit 510 receives a temperature reading710 from the sensor 536 and target focus or bobbin position information720 and applies temperature calibration parameters 730. These can bestored locally on the lens assembly circuit memory 512 as shown in FIG.6. The correction parameters can define a curve or a series of tablevalues associated with given temperature readings that are measuredduring calibration. The process 700 modifies the target Hall sensorvalue (and associated bobbin position) from a base value, based upon thefocus distance provided by the vision system camera to a value thataccounts for the variation of lens focus with respect to lenstemperature. Thus, the base Hall sensor value can be added-to orsubtracted from by the control circuit 510 based upon the prevailingtemperature reading at the lens to generate a temperature correctedtarget value 740.

Likewise, correction for orientation with respect to gravity that canresult in sag or other geometric deformation of the lens membrane indiffering ways is compensated by an (optional) accelerometer 594 thattransmits the spatial orientation 596 of the lens/camera with respect tothe acting directing of gravity to the control circuit via, for example,an I2C protocol. In an embodiment, an orientation correction factor isdetermined (by reading the accelerometer 594), and applied to the targetHall sensor value by the control circuit in a manner similar totemperature correction (FIG. 7) substituting orientation for temperaturein block 710. Since orientation typically remains constant (except inthe case of a moving camera), the determination of orientation can be aone-time event (i.e. at camera setup/calibration), or can occur uponstart up or at a timed interval based upon the control circuit's clock.Like temperature correction, orientation correction parameters cancomprise a curve or lookup table mapped to differing orientations, whichcan be determined during calibration. The appropriate orientationparameter value is applied to the step of determining (574) the targetHall sensor value, and the target value is adjusted to include thisfurther correction for spatial orientation. Note that in the case of amoving camera, the orientation parameter can be continuously updated inthe same manner that temperature is updated to account for changes overtime.

Other parameters (616 in FIG. 6), such as flange-to-sensor distancetolerance, can also be stored in the circuit memory 512. Theseparameters can be updated from the data store of the vision systemcamera upon startup or at another interval of time. The value of eachparameter is used by the control circuit's process to further adjust andcorrect the target Hall sensor value. This overall corrected value isused in the comparison step 578 against the actual measured value tothereby move the bobbin to the correct position.

III. Integrated Time-of-Flight Sensor

FIG. 8 depicts a vision system arrangement 800 and associated visionsystem camera assembly 810 according to an exemplary embodiment. Thecamera assembly 810 can be any acceptable form factor, but typicallyincludes a housing 812 that can be fixedly mounted with respect to alocation, such as a conveyor 820 that moves (arrow 822) objects (e.g.boxes) 830 from a source to a destination. The vision system 800 andassociated camera assembly 810 can be part of a logistics applicationthat finds and decodes IDs printed or adhered to one or more surfaces(e.g. the top surface) 832 of the box 830. While the box has rectangularsurfaces (i.e. is generally rectilinear), it can include sloped orcurvilinear surfaces—for example a triangular-cross section or circularcross-section tube.

The camera assembly 810 can include and internal and/or externalprocessing circuit 812 with associated image processor 814 and memory815 that carries out general image acquisition and image processingtasks using image data received from the image sensor circuitry 816,which is associated with the (e.g.) CMOS image sensor S within theoptics path. The camera circuit can include a focus processor 818, whichcan located in other modules in the overall vision system arrangement,such as the lens assembly 840 (and associated lens control circuit 850),as described generally above (see circuit 350 in FIG. 3. The lensassembly 840 includes a variable lens 842, such as the above-describedliquid lens, that responds to adjustment information (e.g. voltagevalues) 844 to vary its focal distance. The lens assembly 840 caninclude an integrated or external control circuit 850 that canincorporate, or interoperate with, a feedback control circuitarrangement 852, as described generally above. Other circuit elements,such as a memory 854, can also be provided to the lens control circuit850.

Notably, the lens assembly 840 (or another part of the camera assembly810) can include a time-of-flight sensor 860 that is directed to sensedistance (DT) between its emitter/receiver surface and a remote surface,such as the top 832 of a box 830. The time-of-flight sensor in thisembodiment is a commercially available, single point unit, such as modelnumber VL53L0X manufactured by STMicroelectronics of Switzerland, havingthe capability of operating at up to approximately fifty (50) cycles persecond in a fast mode and with an operational range of approximately2000 millimeters and accuracy to within a few centimeters, or less. Useof sensors from other manufacturers and/or other sensor models isexpressly contemplated. As described further below, other models andtypes (differing operational theories) of time-of-flight sensors can beemployed, including multi-point sensors. The time-of-flight sensor isadvantageous in that it is widely available in a variety ofspecifications, compact, relatively low-power, fairly accurate andresistant to many environmental operational conditions. It is alsofairly inexpensive, having a bulk unit cost as low as $1-2 dollars atpresent. The time-of-flight sensor operates by emitting a beam (via anemitter) in which its intensity is modulated at a high frequency, suchthat the emitted beam and the reflected beam (which is received by areceiver portion) exhibit a phase shift therebetween. The degree ofphase shift is measured by the sensor's circuitry, which compares thephase at the emitter with that at the receiver. The measured phase shiftis then converted to a distance measurement based on calibrationparameters that reside within the sensor and/or external electronics.The time-of-flight sensor illuminates the entire scene is captured witha transmitted laser or (e.g. IR-based) light pulse. In some typicalimplementations of a time-of-flight sensor, the beam is emitted at anangle of up to approximately 30 degrees. However emission angles of10-degrees or less can be appropriate for various applications. Thereturned distance value 862 from the sensor can be returned as a voltageor other data form—for example, a digital value—that represents themeasured distance to the object surface. As shown, the sensor can belocated slightly offset from the lens optical axis OA and achieve adesired measurement of the imaged surface. The sensor can beincorporated within the lens housing (viewing the imaged surface througha common front window), or can be located outside the lens housing. Thesensor, can be part of an overall sensor circuit board with appropriateintervening optics to allow it to transmit light to and receive lightfrom the object.

In the depicted, exemplary embodiment, the conveyor 820 generates (e.g.via an encoder) motion information 870 related to conveyor movement thatis transmitted to various processes and processors, including the cameraprocessor 814 and/or an illustrative vision system processor 880. Thevision system processor can be enclosed entirely or partially within thecamera housing, or can be external—for example, instantiated in a PC,laptop, server, handheld device, etc. The exemplary vision systemprocessor 880 can include a variety of functional modules/processors,which perform one or more vision system processes, including visiontools 882, such as edge finders, blob analyzers, pattern recognitiontools, etc. These vision tools 882 can be used to locate variousfeatures in acquired images, such as ID candidate features on a boxsurface. The vision system processor 880 of the exemplary embodimentalso includes ID finding and decoding processes 884 that can identifyand translate found ID candidate features into decoded information 886(e.g. alphanumeric information), that is transmitted over an appropriatecommunication link to other devices and processes, such as a logisticstracking computer and/or conveyor line controller—for example acontroller that starts and stops the line, sounds alarms, gates boxes todiffering destinations based on the ID information. The exemplary visionsystem processor 880 also includes one or more functionalmodules/processors 888 that perform various, object-related processes inaddition to lens autofocus, including region of interest detection,self-triggering of image acquisition/ID-decoding, defect detection andbox size determination.

FIG. 9 depicts a (e.g.) runtime procedure 900 for autofocus of thevariable (e.g. liquid) lens assembly based upon data from thetime-of-flight sensor (860 in FIG. 8). Note that the temperature sensor,which is part of the above-described feedback circuit is read incombination with the initial distance provided by the time-of-flightsensor. As such, the initial runtime focus distance of the variable lensis set based on the measured distance and the measured temperature(which is processed by the above-described feedback circuit). Thisinitial focus process can be performed at startup or on a periodicbasis.

In step 910 of the autofocus procedure 900 an object or objects is/arelocated within the field of view of the camera assembly having anintegrated time-of-flight sensor. The time-of-flight sensor operates bytransmitting light energy to the object surface in step 920 and thislight is reflected to (e.g.) a receiving point on the time-of-flightsensor in step 930. The phase shift between emitted andreflected/received beam is correlated with a distance value (e.g. avoltage value) representing the relative distance from the sensor to theobject surface in step 940.

In step 950, and as described above, the feedback control's focusprocess (532 in FIG. 5) can employ distance data from the time-of-flightsensor to control the focal distance of the lens. The focus process,thus, inputs the sensed distance, and then charges lens focus distanceusing a look-up table and/or an algorithm that employs the distancevalue to vary the input voltage to the (e.g.) liquid lens. This processcan be performed entirely within the on-board circuitry housed withinthe lens barrel. However, sensed distance data can also be provided tothe camera processor 814 (and an optional or alternative on-board focusprocessor 818) and/or vision system processor 880 to perform autofocusand other above-described object processes (block 888). Images of theobject surface are acquired once proper focus is achieved based onoperation of the camera processor and/or vision system processor. Notethat the time-of-flight sensor can be located at the camera sensor imageplane, or at another location of known spacing, from the image plane.The spacing is accounted for in the overall computation of the focusdistance.

In general, the time-of-flight sensor can be set to operate continuouslyat a predetermined cycle rate (e.g. thirty (30) cycles per second). Inmany commercially available sensors, the rate can be varied at leastbetween a so-called slow or accurate mode and a fast mode. In theillustrative embodiment, the fast mode can be employed with reasonableaccuracy. However, the slow mode is also available where greateraccuracy is desired and the cycle rate is still acceptable. Thus, in theprocedure 900 of FIG. 9, the sensor cycles continuously (decision step960) returning a constant distance value equal to the distance from theconveyor (object support base) to the sensor until the next event 970(i.e. the presence of an object in the field of view), at which time anew distance value is presented to refocus the lens (steps 920, etseq.). The lens can again refocus when the distance changes after theobject has passed out of the field of view, or can maintain the samefocus, or can go to a neutral focus setting. More generally, operationof lens autofocus can occur automatically/continuously based oninstantaneous distance readings, or can operate only when objects arewithin, or not within, the field of view. Note that the distancemeasurement by the time-of-flight sensor, or at least the auto-focusingof the lens, should be completed before the subject image is captured.Thus, it is contemplated that there can be general synchronizationbetween the image capture cycle (camera frame rate) and the measurementcycle of the time-of-flight sensor to implement this operation. Apresence detector, such as an object detector (light beam, electric eye,etc.) or an encoder reading can be used to indicate presence of anobject for operation of the vision system camera.

A further form of presence detection can be provided by thetime-of-flight sensor itself. The FIG. 10 shows a self-triggeringprocedure 1000 in which detection of a height variation indicative of apositive object height relative to the conveyor/base surface is used tooperate the vision system camera and trigger image acquisition. In step1010, the time-of-flight sensor operates continuously, taking distancereadings. The circuitry and/or process(or) compares the read distance toprior distance values and/or a baseline value range—for example thecalibrated distance to the conveyor surface (step 1020). When an object(i.e. the object leading edge) enters the camera assembly's field ofview and is picked up by the time-of-flight sensor, the read distancevalue changes (decision step 1030) based on the height of the object.The new distance value is used to establish presence and potentially acurrent object height. The object height is expected to be above aminimum height value (for example, above a baseline height that may bedue to height fluctuations in the conveyor or the presence of debris),and if so then the procedure 100 indicates a trigger event in step 1040and decision step 1050, in which the presence of an object has occurred.The procedure 1000 issues an appropriate trigger signal to the cameraprocessor and/or vision system processor to acquire images of the fieldof view or (where continuous image acquisition occurs) flag the acquiredimages for analysis (step 1060).

With reference to FIG. 11, a procedure 1100 for determining heightand/or size of an object is provided. During calibration, the heightbetween the measurement base (conveyor, etc.) and the camera assemblyimage plane is measured and stored in the vision system. When theobject's leading edge is detected in the field of view by thetime-of-flight sensor, based a change in distance (step 1110), theheight of that edge is determined and the processor(s) monitor(s) motionand height as the object passes through the field of view. Theprocessor(s) tend detect the trailing edge through a second change indistance read by the sensor (step 1120). The locations of the edgetransitions, combined with the degree of motion allow the length of theobject to be computed from which a center can also be computed, and alsoa centroid, taking into account the height (step 1130). Based upon theseparameters, the approximate three-dimensional size and dimensions can beestimated (step 1140).

According to a generalized method, the dimensions of an exemplaryrectilinear object (e.g. a rectangular box) can be determined asfollows: (a) measure distance between the image plane of the cameraassembly and the box in the center of the acquired image; (b) search(using vision system tool) for a rectangular shape in the center of theimage (i.e. top surface of the box); (c) calculate the dimensions of thetop surface, based on the measured distance and the known opticalproperties of imager (sensor pixel array size, focal length, etc.); and(d) calculate the height of the box based on the measured shortestdistance between camera image plane and box surface and the known heightposition of the camera relative to the measurement base (e.g. theconveyor surface, etc.).

In general the width of the object can also be estimated via an acquiredimage or approximated based on length and height (understanding that theconveyor or base defines a maximum width (e.g. 800 millimeters).Optionally, the size parameters can be compared to expected dimensionsbased upon a formula, look-up table of known package dimensions and/orthe consistency of the height (i.e. is the top supposed to be planar?)in decision step 1150. If the object meets expectations, then it can beflagged as acceptable (and this data stored in (e.g.) a trackingdatabase) in step 1160. Conversely if the comparison (decision step1140) determines that one or more measured/estimated parameters deviatefrom acceptable limits, then the object is indicated as possibly (ordefinitely) defective (step 1170). As described further below, themeasurement of dimensions and/or detection of defects can be performedusing a multi-point time-of-flight sensor.

While it is contemplated that various tasks herein can be performedusing a single point time-of-flight sensor, commercially availablesensors can be employed having a one-dimensional (n×1) ortwo-dimensional matrix (n×m) of sensing points that receive reflectedlight from various points of view (typically separated by a given anglethroughout a maximum range (see angle 1218 below). By way of example, an8×8 sensor can be employed. With reference to FIG. 12, a vision systemarrangement 1200 is shown. The camera assembly 1210 in this embodimentincludes an image sensor S and associated optics 1212 located adjacentto (or integrated with) a multi-point time-of-flight sensor 1220. Asexemplary objects 1230 and 1232 pass through (or are presented to) thefield of view 1234, a plurality of sensing points (e.g. points 1240,1242 and 1244) read heights on each object. Other points (e.g. points1250 and 1254) read the height of the base or conveyor 1260, which, whenmoving (arrow 1264), can transmit motion information 1262 to theprocessor(s) 1270 as shown. The matrix of time-of-flight sensing pointsgenerates a stream of one or two-dimensional height profiles that can beoverlaid onto the image data in each acquired frame.

Reference is made to FIG. 13, depicting a procedure 1300 for locatingone or more regions of interest (ROIs) in an image containing one ormore objects (for example objects 1230 and 1232 in FIG. 13). Thisfunction can be used to narrow the search area for ID features, allowingfor more rapid decoding. In step 1310, the multi-point time-of-flightsensor is operated (for example on a continuous, cyclic basis) to take amatrix of height readings within the field of view. The receiveddistance values are associated with a given angle within the field. Theknowledge of this angle allows each point to be overlaid onto theacquired image at an appropriate location so that the given locationwithin the image is assigned a corresponding height value (step 1320).The height values allow the image to be resolved in terms of locationshaving objects present and potentially the tops or sides of objects. Forexample, if adjacent points (e.g. 1240 and 1242 measure the samepositive (above the base 1260) height, then the location is most likelya top surface. Points that measure two different positive heights can beimaging all or part of a side surface. As described above, points thatmeasure the base distance are viewing gaps (1280 in FIG. 12) betweenobjects. Thus, in step 1330, this information is used to determinelocations in each acquired image in which object surfaces (regions ofinterest) are present and locations in the image(s) in which no objector a region of interest (e.g. a side surface) is present (step 1340).With this information, the vision system and ID decoding process can bedirected to focus on such regions of interest. This reduces workload andspeeds the overall ID finding and decoding process as it omits regionsthat are unlikely to contain meaningful (or any) ID information.

A related procedure 1400 is depicted in FIG. 14. Given a plurality ofacquired images in succession, the arrangement 1200 (FIG. 12) can beemployed to determine gap distances between objects. The multi-point (orsingle-point) time-of-flight sensor can determine where edges ofadjacent objects occur based upon the difference between a maximumheight reading (at an object top) and a minimum height reading (at theconveyor/base) in step 1410. The procedure 1400 can compare the locationof the adjacent edges in each images and compare this to the determinedmotion (based on motion encoder information) in step 1420. Based uponthis comparison, the gap distance between adjacent objects can bedetermined with relative accuracy in step 1430. The information can bestored and used in follow-on processes, such as acquisition triggering,region of interest analysis, etc. Notably, by tracking gap locations,the found IDs can be more reliably associated with the proper object inthe image(s).

While the above-described embodiments show a single vision system cameraand associated single-point or multi-point time of flight sensor, it iscontemplated that a plurality of time-of-flight sensors can be used inconjunction with one or more vision system cameras, all (optionally)calibrated to a common coordinate system or frame of reference.

FIG. 15 details another procedure 1500 in which the vision system (e.g.an ID-reading and decoding system) operates in a mode (for example,presentation mode) that only returns a usable result if the object ispresented to the vision system within a predetermined range. That is,objects that pass through the field of view outside of the desired rangeare ignored by one or more steps of the vision system process. In step1510, the object is located within the field of view, based on presencein an image acquired by the sensor and/or a change in distance measuredby the time-of-flight sensor. The time-of-flight sensor determines thedistance of the object (if possible) and returns this distance to theprocessor and its associated process modules in step 1520. The processmodules include a stored distance range that act as a trigger for thereturn of a result to be used in a downstream task (e.g. a logisticstask, object identification, etc. based on a read ID). The procedure1500 compares (in decision step 1530) the returned distance measurementto the trigger range. In essence, the trigger range is adapted to limitthe reading range of the vision system so as to prevent unintentionalreading of an ID on an object. As described above, this can be part of aso-called presentation mode functionality within the vision system. Ifthe distance to the object is within a defined trigger range, then(e.g.) the ID reader (a) captures an image, (b) initiates an ID-dedecoding process, and/or (c) transmits the decoded data to the hostsystem as a result for follow-on tasks (step 1540). If the distance tothe ID code (as indicated by the time-of-flight sensor measurement) isoutside this trigger range, then at least one of the above steps (a)-(c)in the overall ID-reading process can be disabled so a result is notgenerated or stored for downstream use in a task, etc. (step 1550).

IV. Conclusion

It should be clear that superior position correction, on the order of 1millisecond, can be achieved using the local feedback loop instantiatedin a control circuit packaged in the lens assembly. The entire lensassembly package fits within a standard C-mount lens affording a highdegree of interoperability with a wide range of vision system cameramodels and types. The system and method for controlling and correctingthe focus of a liquid (or other similar auto-focusing) lens describedherein can be employed rapidly, and at any time during camera runtimeoperation and generally free of burden to the camera's vision systemprocessor. This system and method also desirably accounts for variationsin focus due to thermal conditions and spatial orientation (i.e. lenssag due to gravity). This system and method more generally allow for alens assembly that mounts in a conventional camera base.

The use of an integrated single-point or multi-point time-of-flightsensor in conjunction with a vision system camera arrangement canprovide a variety of useful functions, such as autofocus,self-triggering, region of interest determination, controlling readingrange (e.g. for use in ID-reader presentation mode), and/or objectsize/quality analysis. The wide availability, relatively low cost,reasonable speed and accuracy of such sensors renders them desirable fora variety of applications and allows their use on either a camerahousing or interchangeable lens assembly.

The foregoing has been a detailed description of illustrativeembodiments of the invention. Various modifications and additions can bemade without departing from the spirit and scope of this invention.Features of each of the various embodiments described above can becombined with features of other described embodiments as appropriate inorder to provide a multiplicity of feature combinations in associatednew embodiments. Furthermore, while the foregoing describes a number ofseparate embodiments of the apparatus and method of the presentinvention, what has been described herein is merely illustrative of theapplication of the principles of the present invention. For example,while a Hall sensor is used to measure position, a variety of alternateposition-sensing devices can be used in association with the feedbackloop herein. For example an optical/interference-based position sensorcan be employed in alternate embodiments. Also, it is contemplated thatthe principles herein can be applied to a variety of lenses (liquid andotherwise), in which the curvature of the lens is varied via electroniccontrol. Thus the term “variable lens assembly” should be taken broadlyto expressly include at least such lens types. In addition while variousbobbin position corrections are performed within the lens controlcircuit and feedback loop, it is contemplated that some corrections canbe performed within the vision system camera processor, and thecorrected focal distance is then sent to the lens assembly for use infurther feedback loop operations. As used herein, various directionaland orientation terms such as “vertical”, “horizontal”, “up”, “down”,“bottom”, “top”, “side”, “front”, “rear”, “left”, “right”, and the like,are used only as relative conventions and not as absolute orientationswith respect to a fixed coordinate system, such as gravity. Accordingly,this description is meant to be taken only by way of example, and not tootherwise limit the scope of this invention.

What is claimed is:
 1. A vision system comprising: a 2D image sensorthat acquires images of a scene; an imager lens that projects receivedlight from the scene onto the image sensor; a time-of-flight sensor thata receives a distance measurement from the scene; and a processorarranged to receive distance data from the time-of-flight sensor, andbased upon the distance data, determine one or more height measurementswith respect to one or more objects within the scene so as to perform ananalysis process on the one or more objects.
 2. The vision system as setforth in claim 1 wherein the one or more objects define a rectilinearshape and the analysis process defines measuring dimensions of the oneor more objects by (a) measuring a distance between the image sensor andthe object in the center of the image; (b) searching for a rectangularshape in the center of an image acquired by the image sensor with one ormore vision system tools, (c) computing dimensions of a top surface ofthe one or more objects relative to the rectangular shape, based on themeasured distance and known optical properties of image sensor andimager lens, and (d) calculating a height of the one or more objects,based on a measured shortest distance between the image sensor and thetop surface and a known height position of the image sensor based upon areference surface.
 3. The vision system as set forth in claim 2 whereinthe reference surface comprises a moving conveyor that passes the one ormore objects through the scene.
 4. The vision system as set forth inclaim 3 wherein the processor is arranged to determine deviations fromthe rectilinear shape to determine a defect in the one or more objects.5. The vision system as set forth in claim 2 wherein the one or moreobjects are in relative motion with respect to the image sensor and thetime-of-flight sensor.
 6. The vision system as set forth in claim 5wherein the image sensor and the imager lens are mounted to acquireimages image the one or more objects transported on a conveyor through ascene and are operatively connected to an ID decoder that finds anddecodes ID information on the one or more objects.
 7. The vision systemas set forth in claim 6 wherein the time-of-flight sensor is asingle-point time-of-flight sensor.
 8. The vision system as set forth inclaim 1 wherein the time-of-flight sensor is a multi-pointtime-of-flight sensor.
 9. The vision system as set forth in claim 8wherein the processor receives a plurality of height values concurrentlyfrom the multi-point time-of-flight sensor relative to discrete parts ofthe scene, and based on the measured height values defines at least oneregion of interest with respect to the one or more objects.
 10. Thevision system as set forth in claim 9 further comprising an ID decoderthat receives image data from the scene and locates and decodes IDinformation within the at least one region of interest.
 11. The visionsystem as set forth in claim 9 further comprising an ID decoder thatreceives image data from the scene and locates and decodes IDinformation from the image data, and wherein the processor is arrangedto detect a plurality of objects based on the height values in the sceneand to associate one or more located IDs relative to respective of theplurality of objects.
 12. The vision system as set forth in claim 1wherein the imager lens defines a removable lens assembly that includesan electrically controlled variable lens and a lens controller circuitoperatively connected to the variable lens, the lens controller circuitbeing housed in the removable lens assembly.
 13. The vision system asset forth in claim 12 wherein the time-of-flight sensor is containedintegrally within the removable lens assembly and interconnected to thelens controller circuit.